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The following describes research supported by the 
Teagle Foundation for the development and testing of 
an instrument for measuring college student perceptions 
of institutional mission in the learning environment. 
Findings from reliability and validity testing of a set of 
questions called the Mission Perception Inventory (MPI) 
are discussed. The MPI was first developed in 2004 from 
qualitative analysis of selected mission statements of 
postsecondary institutions in the United States. Questions 
were formulated around the construct of mission 
perception, operationally defined as an individual's 
judgment of his or her engagement in the learning 
environment of an institution as defined by its mission. 
Testing was performed on data from four consecutive 
annual administrations of the MPI appended as consortia 
questions to the National Survey of Student Engagement 
(NSSE). Data were obtained from a sample of men and 
women in the first and senior college years (N = 18,671). 
Results from reliability and factor analysis of the data 
support the construct validity and reliability of the 
scale by year in terms of stability, internal consistency, 
and item-total correlation matrices. This Professional 
File provides consistent evidence that the MPI is a valid 
and reliable measure overtime in a changing student 
population. Hence, this instrument can be a useful 
outcomes measure for activity focused on advancing 
institutional mission on college campuses. 
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Introduction 

The mission statement of a college or university 
assigns purpose and definition to all the activity 
that supports and engages students in the learning 
environment. An institution's goals and objectives 
are suitably derived from its mission statement, 
and it is important for an institution to have clearly 
articulated objectives for student learning and 
development (Chickering, 1993). Crafting effective 
methods for measuring the relationship between 
student learning objectives and student learning 
outcomes is critical to institutional assessment, 
and this research looks for links in the environment 
where learning occurs. The assessment tool 
used is the Mission Perception Inventory (MPI), 
an instrument designed to measure student 
perception of institutional mission manifest in the 
learning environment. 

The assumptions of this research are, one, 
measures of student perception of the learning 
environment in relation to mission constructs can 
be devised and, two, scores on these measures 
are valid indicators of student perception of the 
institutional mission apparent in the learning 
environment. The instrument, MPI, is one question 
set shorter but similar in design to the National 
Survey of Student Engagement (NSSE) and includes 
questions on student perceptions of college 
mission. The NSSE was designed to act as a process 
indicator to help colleges and universities see 
connections between programs and activities and 
student learning outcomes (Pike, Kuh, & Gonyea, 
2003). The MPI is also a process indicator, in this 
case, assessing activity meant to enhance the 
learning environment and, at the same time, the 
context where learning occurs. The validity and 
reliability testing performed here is an investigation 
of the ability of the MPI to assess and reflect the 
learning environment. 

An instrument developed and tested on 
one population at one time may not produce 
comparable results in subsequent trials on 
changing populations. Reliability analysis can 


assure confidence in the question items, and factor 
analysis can identify fundamental constructs within 
a set of items. According to Pike (1 995), reliability 
is the degree to which a set of items measures 
the same thing consistently across respondents. 

The ability of the MPI to produce consistent 
results needed to be tested over time to confirm 
instrument reliability and keep confidence in results 
from MPI administration. 

The intent of this research is to analyze and 
interpret results from further psychometric testing 
of a mission perception instrument following its 
administration to independent samples at different 
points in time. This Professional File describes 
tests on data collected from the administrations 
of the instrument and presents the results. Along 
with reliability testing, factor analysis assesses 
construct validity of the MPI scale. Additionally, 
the hypothesis is tested that comparable results 
for stability, internal consistency, and item-total 
correlation matrices will be found following analysis 
of the successive years'data from administrations of 
the MPI. 

Review of the Literature 

Higher education researchers hunger for 
fresh, reliable ways to gather and report data on 
student learning outcomes. This is particularly true 
today as a consequence of escalating stakeholder 
demands for accountability (Ewell, 2007), and a 
rise in new and unique assessment challenges 
that may be unmet by the limits of available tools. 
Some examples of key goals articulated in college 
and university mission statements currently are 
(a) provide professional preparation, (b) develop 
leadership skills, and (c) foster respect for diversity. 
These kinds of mission goals are realized in the 
programs and activities of the student learning 
environment, and assessing student perceptions of 
that environment is examined here. 

For almost two decades, researchers have 
explored whether institutional characteristics, 
apart from mission, have an influence on student 
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outcomes. Weidman (1 989) found that institutional 
characteristics do matter to student learning, 
but further research by Pascarella and Terenzini 
(1991), which controlled for student background, 
determined the relationship to be weak at best. 
Toutkoushian and Smart (2001 ) suggest that 
although institutional characteristics may not 
directly correlate with student learning, a student's 
experiences within the institutional environment do 
correlate strongly with outcomes. A methodology 
developed by Pike et al. (2003) has been successful 
for studying the relation of institutional mission to 
student engagement and learning, and it is similarly 
employed here. 

Chickering (1 993) points out the importance for 
an institution to have clearly articulated objectives 
for student learning and development. Ideally, the 
objectives pervade the institution's programs and 
climate and are widely shared and emphasized in 
oral and written communication. A strong sense 
of college mission that unifies the educational 
experience of students can reduce ambiguity and 
define purpose for students and others in the 
institution. For example, some of the constructs 
embodied in the mission statements of institutions 
participating in this research include: education 
as empowerment; a community of learning; moral 
character; leadership in service; career preparation; 
and appreciation for diversity. 

In America's Best Colleges (2011), the US. 

News and World Report annual guide to colleges, 
institution mission is described according to its 
Carnegie Classification, but that categorization is 
not followed here. For this research, "mission" is 
operationally defined as the overall purpose and 
activity of an institution as defined by the goals of 
the mission statement. This allows for flexibility in 
characterizing the institutions in the consortia and 
understanding the activities they engage in to fulfill 
their educational mission. 

Research by Pike et al. (2002) on institutional 
mission concludes that mission constructs can be 
measured. The methodology NSSE uses to produce 


benchmarks, or scales, of "effective educational 
practice" (Kuh, 2001, p. 13) is reflected here in the 
way factor scales have been produced. 

The purpose of the factor analytic method used 
here is to evaluate score validity once the measure 
had been developed and direct focus on whether 
scores are valid and the MPI measures "the correct 
something" (Thompson, 2004, p. 4). Tests used 
in this research are meant to clarify operational 
constructs for an area, in this case, the measurement 
of student perception of institutional mission. The 
scores, or outcomes, of those measures can then 
be a means for evaluating students' perceptions 
of mission-related curricular and co-curricular 
programs on campus. 

Thompson and Daniel (1 996) state that 
factorial validity of a test is "given by its loadings in 
meaningful, common, reference factors" (p. 197). 

The Rotated Factor Matrix produced from initial 
testing of the instrument (Boylan, 2005) compared 
favorably against that standard for validity. 

Of the two main types of factor analysis, 
exploratory and confirmatory, the former has 
been employed in this research. Exploratory 
factor analysis uncovers the structure of a set of 
variables by grouping variables that are correlated, 
particularly in the case when no hypothesis about 
the nature of the underlying factor structure of 
their measures has been posited (Goho & Blackman, 
2009). To determine the statistical validity of these 
items, confirmatory factor analysis should be 
performed once the factor structure and variable 
loadings can be identified in advance (Coughlin, 
2005; Pedhazur & Schmelkin, 1991). This study 
describes how, over the course of four years, as 
more institutions participated and the number of 
respondent cases increased, three of the four factors 
found in exploratory analysis were confirmed by 
factor analysis performed on MPI results from 
successive administrations. 

A limitation of this study is reliance on research 
conducted in closed consortia of Catholic colleges 
and universities participating in one or more 
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NSSE administrations from 2004 through 2007. 
Although the results may not be generalizable to all 
postsecondary institutions, the methodology may 
be adapted to explorations of institutional mission 
at other colleges. 

In anticipation of future opportunities for testing 
the construct validity of the MPI for use by other 
colleges, it was purposely intended at the question 
construction stage of this research to develop 
mission questions that are "institution affiliation- 
neutral." Despite using the mission statements of 
the first 1 4 consortium institutions, all Catholic, 
in a qualitative analysis to draw out common 
constructs for MPI questions, the constructs that 
became questions are not exclusive to Catholic 
institutions. Constructs like "academic excellence" 
and "leadership development" are familiar in most 
postsecondary institutions' mission statements. 
Further, the opportunity to test the questions on 
two consortia simultaneously, one Catholic and 
one private, did arise in 2008, thanks to a Teagle 
Foundation grant for this research. Preliminary 
factor analysis suggests that the inventory is equally 
reliable for both consortia. In fact, the investigation 
of reliability has been repeated with data from 
a subsequent year administration of the MPI on 
two consortia with different affiliations. Additional 
results are available from the author. 

Methodology 

The process of developing and validating 
the instrument used a mixed method qualitative 
and quantitative research analysis. The content 
of question items was initially developed from 
a qualitative analysis of mission statements 
of colleges registered to participate in a NSSE 
consortium proposed by the research administrator. 
To facilitate later opportunities for analysis with 
NSSE individual items and benchmarks, the same 
scale-format used for those questions is employed 
for the mission research questions. Draft mission 
question items using this format were circulated 
in advance among consortium participants for 


feedback, and then revised and checked for face 
validity with experts in survey research at the NSSE 
headquarters. The instrument was then given to 
a pilot group of subjects to check question clarity, 
and final revisions followed. 

Since the research administrator has established 
relations with personnel at many Catholic 
colleges, it was most convenient to obtain the 
participation of a group of Catholic colleges in 
the mission research project. The willingness of 
Catholic colleges to participate also stemmed from 
shared mutual interest in measuring institutional 
mission effectiveness; however, that interest is not 
exclusive to Catholic colleges. Providing evidence 
of mission effectiveness to some degree is a 
requirement of all regional accreditation bodies for 
all colleges, whether private or public, religiously 
affiliated or not. Therefore, when developing the 
mission questions, it was a deliberate choice to 
design the questions so they could be employed 
for assessment purposes by any postsecondary 
institution of any affiliation. 

In the 2004 inaugural year of this study, the 
mission research instrument was appended to 
the NSSE main survey and administered by 1 4 
Catholic colleges in a consortium. The 2004 data (N 
= 3,605) were released by participating institutions 
and obtained to begin this research. This process 
of survey administration and obtaining data was 
repeated in each of the following three years, 2005 
(N = 2,448), 2006 (N = 4,897), and 2007 (N = 7,721 ). 

Table 1 displays the MPI question items used by 
year. In any given year, a consortium can append a 
total of 20 question items to the NSSE. Questions are 
static or may vary. In addition to the demographic 
question on "current religious preference," 14 items 
have been the same every year 2004 through 2007. 
The five questions that varied did so because of 
changes in consortia coordinators by year. Note that 
in 2004 and 2007, however, the items are the same, 
allowing the opportunity to explore confidently the 
strength of that joint MPI dataset. Also, in successive 
years 2005, 2006, and 2007, 1 6 items are the same, 
and that is explored, too. 
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Table 1 

MPI Scale Items and Reliability by Year 



2004 

2005 

2006 

2007 

Current religious preference: (Mark one) (Not included in MPI) 

X 

X 

X 

X 

The mission of this institution is widely understood by students. 

X 

X 

X 

X 

Ethical and spiritual development of students is an important part of the mission at this institution. 

X 

X 

X 

X 

This institution offers opportunities for volunteering and community service. 

X 

X 

X 

X 

Social and personal development of students is an important part of the mission at this institution. 

X 

X 

X 

X 

This institution offers opportunities for developing leadership skills. 

X 

X 

X 

X 

The heritage of the founding religious community of this institution is evident here. 

X 

X 

X 

X 

The faculty, staff, and students here are respectful of people of different religions. 

X 

X 

X 

X 

The faculty, staff, and students here are respectful of people of different races and cultures. 

X 

X 

X 

X 

People of different sexual orientations are accepted socially here. 

X 

X 

X 

X 

Students feel free to express their individual spirituality here. 

X 

X 

X 

X 

The environment here encourages students to develop an appreciation of diversity. 

X 

X 

X 

X 

The professors at this institution discuss the ethical implications of what is being studied. 

X 

X 

X 

X 

Within the past week, have you spent time in private prayer or meditation? 

X 

X 

X 

X 

Within the past week, have you participated in a religious service? 

X 

X 

X 

X 

The mission of this institution is reflected in its course offerings. 


X 

X 

X 

As a result of your experience here, you are more aware of your own personal values. 


X 

X 

X 

At this institution, there are opportunities for students to strengthen their religious commitment. 

X 



X 

*How important is it to you that you accomplish the following objectives? Raising a family. 





*How important is it to you that you accomplish the following objectives? Becoming an authority in your field. 





How important is it to you that you accomplish the following objectives? Volunteering in community service. 

X 



X 

How important is it to you that you accomplish the following objectives? Influencing social values. 

X 



X 

Since you have been here, has your religious commitment become: stronger 


X 

X 


Frequency on campus: Discussion about moral and ethical issues 


X 

X 


Number of MPI scale items 

17 

18 

18 

19 

Total Scale a = 

0.880 

0.894 

0.902 

0.907 

N- 

3,605 

2,448 

4,897 

7,721 


/Vote: *ltem deleted 2004 


To assess the construct validity of the MPI, 
the items of the instrument used by year were 
subjected to exploratory factor analysis to yield a 
factor structure containing the best performing 
subset of items (Chesney, Neilands, Chambers, 
Taylor, & Folkman, 2006). Reliability of the overall 
MPI was measured via internal consistency 


coefficient alpha (Cronbach, 1951). A complete 
description of the statistical analysis approach is 
described previously (Boylan, 2005). 

In addition to listing the question items, Table 
1 shows by year the results of internal consistency 
analysis (Cronbach's alpha) for the total MPI scale, 
not accounting for the presence of any internal 
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factors. The strongest Cronbach alpha score by year 
is indicated for the MPI 2007 ( a = .907), but scores for 
every year exceed .80. Therefore, all MPI scales are 
reliable by year. However, are all scales equally valid 
for measuring mission perception? Which set of 
items should comprise the MPI to produce the most 
consistent and reliable measure in future testing? 

Results 

The research is based on data from first-year 
and senior students at 54 institutions participating 
over four years in the NSSE 2004-2007 Catholic 
College Consortia. Each year, the NSSE main survey 
was administered to the consortium sample with 
an added 20-item set of consortium questions. 

There were 1 8,671 valid cases obtained over four 
administration years. Respondents were classified 
by schools as either first-year students or seniors, 
and in truth, there was a roughly 50/50 division 
each year. Of total respondents for the four years, 
about 27% are male and 73% female. 


Figure 7. Percent distribution institutions by regions 
2004-2007. 

Figure 2. Consortia institutions by urbanicity 2004- 
2007. 


Institutional characteristics by regional location 
and urbanicity are shown in Figures 1 and 2. Figure 
1 shows the percent distribution of participating 
consortia colleges by region of the United States, 


with most from the North (43%), and in descending 
order from the Midwest (31%), South (17%), and 
West (9%), a fair reflection of overall institutional 
distribution nationwide. As indicated in Figure 2, 
54% of institutions participating in consortia data 
sharing were from suburban areas, 31 % from urban, 
and 15% from rural. 

Factorial validity 

The sample met the subject to variable ratio 
of 5:1 (Tabachnick & Fidell, 1 996). To address the 
research question about validity and reliability of 
the MPI, a Cronbach's alpha test was conducted 
on the total set of mission question data from 
administration years 2004, 2005, 2006, and 2007, 
individually. For each year, question items with 
low inter-item correlation, or lower than .30 (p. 

641), such as"becoming an authority in my field" 
and "raising a family" from the first administration 
in 2004, and others about drinking and cheating 
behavior from 2005 and 2006, were removed due 
to low factor loading in exploratory factor analysis. 
Repeating the steps of analysis and removing 
items that appear in the pattern matrix as having 
low item-total correlations, or below .32 (p. 677) 
produced the most reliable set of items for MPI 
scales for each year of data. 

The next object of analysis was to develop the 
most reliable MPI scale using the greatest number 
of question items shared over four years. Results are 
shown in Table 2. The text of questions and their 
item factor loadings by aggregated dataset are 
indicated. Data were aggregated in sets: two-year 
(2004/2007), three-year (2005-06-07), and four-year 
(2004-05-06-07). Sets were created by combining 
results from question inventories that shared the 
largest number of questions in common. In each 
dataset, about 6% to 8% of cases are classified as 
missing, an acceptable level. 

Exploratory factor analysis was conducted on 
each aggregate dataset, and low-loading items 
were dropped. The factor analysis produced three 
factors from the combined MPI 2004/2007 data, 
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Table 2 

Results of Factor Analysis ofMPI by Combined Data Years 


Factors 

2-Year 

3-Year 

4-Year 

2004 and 2007 

2005-06-07 

2005-06-07 

2004-2005-2006-2007 

1 

2 

3 

1 

2 

3 



1 

2 

3 

Current religious preference: (Mark one) 
(Not included in MPI) 












The mission of this institution is widely understood 
by students. 

.518 



.555 





.542 



Ethical and spiritual development of students is an 
important part of the mission at this institution. 

.693 



.681 





.684 



This institution offers opportunities for volunteering 
and community service. 

.687 



.584 





.653 



Social and personal development of students is an 
important part of the mission at this institution. 

.638 



.677 





.733 



This institution offers opportunities for developing 
leadership skills. 

.633 



.617 





.679 



The heritage of the founding religious community 
of this institution is evident here. 

.635 



.593 





.613 



The faculty, staff, and students here are respectful 
of people of different religions. 


.703 



.442 

.698 




.675 


The faculty, staff, and students here are respectful 
of people of different races and cultures. 


.698 



.478 

.648 




.683 


People of different sexual orientations are accepted 
socially here. 


.680 



.688 





.694 


Students feel free to express their individual 
spirituality here. 


.692 



.722 





.694 


The environment here encourages students to develop 
an appreciation of diversity. 


0.664 



0.709 





0.677 


The professors at this institution discuss the ethical 
implications of what is beinq studied. 

.505 



.534 





.489 



Within the past week, have you spent time in private 
prayer or meditation? 



.340 



<.32 





.696 

Within the past week, have you participated in a 
reliqious service? 



.308 



<32 





.642 

The mission of this institution is reflected in its course 
offerinqs. 

.518 



.624 






n/a 


As a result of your experience here, you are more aware 
of your own personal values. 

n/a 



.565 






n/a 


At this institution, there are opportunities for students 
to strenqthen their reliqious commitment. 

n/a 




n/a 





n/a 


*How important is it to you that you accomplish the 
followinq ? Raisinq a family. 

n/a 




n/a 





n/a 


*How important is it to you that you accomplish the 
followinq? Becominq an authority in your field. 

n/a 




n/a 





n/a 


How important is it to you that you accomplish the 
followinq objectives? Volunteerinq in community service. 




.726 

n/a 





n/a 


How important is it to you that you accomplish the 
followinq objectives? Influencinq social values. 




.720 

n/a 





n/a 


Since you have been here, has your religious 
commitment become: stronqer (2005-06 only) 

n/a 




n/a 





n/a 


Frequency on campus: Discussion about moral 
and ethical issues (2005-06 only) 

n/a 




n/a 





n/a 


Scale items 

17 

16 

14 

14 

Total Scale a = 

.887 

.903 

.911 

.883 

N= 

11,326 

15,066 

15,066 

18,671 

variance explained 

46.71 

48.45 

50.93 

50.55 


Note: Asterisk indicates low loading of item in Exploratory Factor Analysis; therefore, item removed. 
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three from the MPI 2005-06-07 data, and three from 
the 2004-05-06-07 data. In the three-year analysis 
for years 2005-06-07, evidence suggests that two 
low-loading items were possibly confounding the 
alignment of two others that previously aligned 
strongly with the diversity factor. So, low-loading 
items were removed and factor analysis was 
conducted an additional time on the three-year 
dataset. As a result, two strong factors, or subscales, 
with very good to excellent loadings of all 1 4 items 
were produced. These scales are consistent with 
ones previously identified as Sense of Mission (9 
items) and Respect for Diversity (5 items). 

To interpret the results of factor analysis shown 
in Table 3, the rule of thumb is followed wherein 
only variables with loadings of .32 and above are 
interpreted (Tabachnick & Fidell, 1 996). The greater 
the loading, the more the variable is a pure measure 
of the factor. Comrey and Lee (1 992) suggest that 
loadings in excess of .71 are considered excellent, .63 
very good, .55 good, .45 fair, and .32 poor. 

With regard to subscales shown in Table 3, the 
most reliable in every administration year are Sense 
of Mission and Respect for Diversity. Other scales 
previously identified, Individual Actions in 2004 and 
Religious Practice in 2004, 2005, 2006, and 2007 
(Boylan, 2005), are also shown in Table 3. These 


Table 3 

Cronbach's Reliability Analysis of the 
Mission Perception Inventory (MPI) and 
Subscales 2004-2007 


Subscales 

2004 

2005 

2006 

2007 

Sense of Mission 

.87 

.88 

.88 

.90 

Respect for Diversity 

.85 

.84 

.87 

.86 

Individual Actions 

.67 

n/a 

n/a 

n/a 

Religious Practice 

.62 

.55 

.54 

.62 

MPI scale 

.88 

.89 

.90 

.91 


subscales have few variables, just two or three by 
year. The Religious Practice factor was consistent 
over four years. Thereto re, that scale could be 
employed by consortia in the future, along with the 
Sense of Mission and Respect for Diversity scales, 
if that is of interest to participating institutions. 
However, caution is advised when interpreting 
factors, like Religious Practice, that consist of just 
two items (Tabachnick & Fidell, 1 996). 

Discussion 

Findings from testing of the MPI suggest that 
the overall scale and subscales are reliable over 
time, are supported by factor analysis, and are of 
relevance to postsecondary institutions interested 
in gathering evidence of student perception of 
mission. 

To summarize, the MPI was administered 
annually for four consecutive years by a consortia 
of varying college participants, and the data were 
collected. Testing of the instrument, with poorly 
correlated items by year removed, was conducted, 
and the instrument was found overall to be a 
reliable and valid measure of student perception of 
institutional mission. As shown in Table 3, additional 
testing via factor analysis produced up to four 
reliable subscales from the MPI called Sense of 
Mission, Respect for Diversity, Individual Actions, 
and Religious Practice. Because the Individual 
Actions and Religious Practice scales have so few 
items each, only Sense of Mission and Respect for 
Diversity, with more items and higher reliability, 
would be most useful to subsequent survey 
administrations that might be conducted to explore 
student perception of mission. 

It is important to consider the replicability, 
utility, and complexity of factors when interpreting 
output. Here, by selecting the fewest factors to 
comprise the MPI, the utility of the inventory can 
be increased. The high internal consistency of 
items and strong correlations within factors lends 
confidence to the results of future administration of 
the MPI. 
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Conclusions and Implications 

The subscales produced from analysis by 
administration year indicate the presence of 
those concepts in students' perceptions of their 
educational experiences at consortium colleges. The 
reliability of the overall MPI scale indicates that it 
can be employed with confidence as a measure of 
student mission perception in subsequent studies. 
Also, the high reliability of the Sense of Mission and 
Respect for Diversity subscales indicates that these 
scales can be used alone or in concert with other 
instruments to measure those constructs. 

Another appropriate research exploration 
using the MPI and subscales of Sense of Mission 
and Respect for Diversity is to administer them to 
different consortia of institutions with different 
input characteristics, and compare results. Indeed, 
that analysis has been performed on results from 
the NSSE 2008 and 2009 administrations, in which 
two consortia of institutions, one Catholic and one 
private, administered the same MPI questions to 
their first- and senior-year students. See results 
at http://www.marywood.edu/instresearch/ 
professional-publications-activities.html 
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